Search Results: "ashley"

28 April 2008

David Pashley: User Administration under PostgreSQL 8.3

A while ago I published an article on PostgreSQL user administration. Typically, things have changed since I wrote that article. I thought I'd detail a couple of the differences since I wrote that guide. The major difference is that you now have roles rather than users and you use the CREATE ROLE command to create them instead of CREATE USER, although the latter command still works. The command line options for the createuser command have changed as a result too. Before superuser and the ability to create new users were the same thing. Now you can give a role permissions to create new roles without giving them superuser powers. The options are now -s for superuser and -S for not superuser, -d to allow them to create databases and -D to disallow database creation and -r to allow the new role to create other roles and -R to prevent them. for a standard user you probably want somethig like:

createuser -S -D -R -P user

The -P makes createuser ask you for a password for the new role. You can find out more information about the new role system in PostgreSQL in the user management and CREATE ROLE reference sections of the manual.

Read Comments (1)

26 April 2008

David Pashley: Upgrading to latest Pyblosxom

I'm currently upgrading my blog to PyBlosxom 1.4.3. I apologise for any broken links or entry flooding. Update: I've finished playing now. I've upgraded to 1.4.3 and I don't think I've broken anything yet. I've also taken the opportunity to add a couple of plugins to add tagging to entries and added the obligatory tag cloud to the side bar rather than the list of months. I'm going to make some changes to the comment plugin later to add OpenID support. I'd be interested to know of any other pyBlosxom plugins you find useful. I did manage to make a mistake by using vim to edit entries to add some tags rather than my wrapper script to keep timestamps the same. This is where I'm glad I have a database table with the metadata from all my entries to hand. A quick touch foo.txt -d 2006-06-07 19:02:57+01 later and everything was fixed. Hopefully not too many people got bitten by the few entries that had new dates for a few minutes. Please let me know if you notice anything broken.

Read Comments (0)

David Pashley: Violating Perl Module Namespaces

Perl doesn't enforce access to modules' namespaces. This would usually be considered a bad thing, but sometimes it allows us to work around problems in modules without changing their code. Here's a perfect example: I've been writing a script to talk to an XML-RPC endpoint, using Frontier::Client but for one of the requests, the script throws the following error:

wanted a data type, got  ex:i8'

Turning on debugging showed the response type was indeed ex:i8, which isn't one of the types that Frontier::Client supports.

<?xml version="1.0" encoding="UTF-8"?>
<methodResponse xmlns:ex="http://ws.apache.org/xmlrpc/namespaces/extensions">
  <params>
    <param>
      <value>
        <ex:i8>161</ex:i8>
      </value>
    </param>
  </params>
</methodResponse>

Searching through the code shows Frontier::Client is a wrapper around Frontier::RPC2 and the error message happens at the following section:

     elsif ($scalars $tag )  
       $expat-> 'rpc_text'  = "";
       push @  $expat-> 'rpc_state'   , 'cdata';
     else  
       Frontier::RPC2::die($expat, "wanted a data type, got \ $tag'\n");

So we can see that it's looking up the tag into a hash called %scalars to see if the type is a scalar type, otherwise throws the error we saw. Looking at the top, we can see this hash:

%scalars = (
    'base64' => 1,
    'boolean' => 1,
    'dateTime.iso8601' => 1,
    'double' => 1,
    'int' => 1,
    'i4' => 1,
    'string' => 1,
);

So, if we could add ex:i8 to this scalar, we could fix the problem. We could fix the module, but that would require every user of the script to patch their copy of the module. The alternative is to inject something into that hash across module boundaries, which we can do by just refering to the hash by it's complete name including the package name. We can use:

$Frontier::RPC2::scalars 'ex:i8'  = 1;

Now when we run the script, everything works. It's not nice and it's dependent on Frontier::RPC2 not changing. but it allows us to get on with our script.

Read Comments (1)

24 April 2008

David Pashley: Violating Perl Module Namespaces

wanted a data type, got  ex:i8'

Turning on debugging showed the response type was indeed ex:i8, which isn't one of the types that Frontier::Client supports.

<?xml version="1.0" encoding="UTF-8"?>
<methodResponse xmlns:ex="http://ws.apache.org/xmlrpc/namespaces/extensions">
  <params>
    <param>
      <value>
        <ex:i8>161</ex:i8>
      </value>
    </param>
  </params>
</methodResponse>

Searching through the code shows Frontier::Client is a wrapper around Frontier::RPC2 and the error message happens at the following section:

     elsif ($scalars $tag )  
       $expat-> 'rpc_text'  = "";
       push @  $expat-> 'rpc_state'   , 'cdata';
     else  
       Frontier::RPC2::die($expat, "wanted a data type, got \ $tag'\n");

So we can see that it's looking up the tag into a hash called %scalars to see if the type is a scalar type, otherwise throws the error we saw. Looking at the top, we can see this hash:

%scalars = (
    'base64' => 1,
    'boolean' => 1,
    'dateTime.iso8601' => 1,
    'double' => 1,
    'int' => 1,
    'i4' => 1,
    'string' => 1,
);

$Frontier::RPC2::scalars 'ex:i8'  = 1;

Now when we run the script, everything works. It's not nice and it's dependent on Frontier::RPC2 not changing. but it allows us to get on with our script.

Read Comments (1)

David Pashley: Photography In Public Areas Early Day Motion

I just emailed my MP the following letter:

Dear David Lepper, I would just like to thank you for signing Auston Mitchell's Early Day Motion 1155 Photography In Public Areas. I have been increasingly concerned with reports of police action against innocent photographers, including most recently a man assaulted by several security guards in Stoke (http://www.flickr.com/photos/happyaslarry/2420960125/). I'm sure you appreciate Brighton's reputation as an artistic city and your support for this motion shows your continued support for the photography community in Brighton. Yours sincerely, David Pashley

If your MP hasn't signed this EDM, I recommend you contact them to urge them to sign it and if they have, contact them again to thank them.

Read Comments (0)

17 April 2008

David Pashley: Using In-memory tarballs with Archive::Tar

Archive::Tar is a useful library for working with tar archives from Perl. Unfortunately, one thing it doesn't allow is using data from memory as the archive. From the TODO section:

Allow archives to be passed in as string Currently, we only allow opened filehandles or filenames, but not strings. The internals would need some reworking to facilitate stringified archives.

Fortunately, it does allow you to use a filehandle. I've previously mentioned about how useful the IO::Handle subsystem in perl is. And we should be able to use it in this case. The module we'll want is IO::String, which is a IO::Handle over a perl scalar. We can use it:

my $tar = new Archive::Tar(new IO::String($data));

Unfortunately when we run this now we get:

Cannot read compressed format in tar-mode at Foo.pm line 41
No data could be read from file at Foo.pm line 41

It turns out that this is because Archive::Tar uses IO::Zlib internally if the file isn't uncompressed, but this doesn't provide the ability to uncompress from a filehandle. The answer is to uncompress the data before passing it to Archive::Tar and the easiest way to do this is to use the IO::Uncompress::Gunzip module, so we can rewrite our code to:

my $tar = new Archive::Tar(new IO::Uncompress::Gunzip(new IO::String($data)));

Now when you run the script, Archive::Tar has an uncompressed tar stream. Yet another situation where IO::Handles comes to the rescue.

Read Comments (1)

David Pashley: Boilerplate code for a perl class

Because I always forget when I need to create a new class in perl:

package Foo::Bar;
use strict;
use warnings;
sub new  
   my $this = shift;
   my $class = ref($this)   $this;
   my $self =  ;
   bless $self, $class;
   $self->initialize(@_);
   return $self;
 
sub initialize  
   my $self = shift;
 
1;

If you have any useful additions I'd love to know.

Read Comments (4)

3 April 2008

David Pashley: InnoDB being silently disabled

Regular viewers will know that I don't think favourably of MySQL. Here is yet another reason. Let's create an InnoDB table:

mysql> CREATE TABLE  User_  (
mysql> ...
mysql> ) ENGINE=InnoDB DEFAULT CHARSET=latin1;
Query OK, 0 rows affected, 1 warning (0.04 sec)

One warning, but we're running this as part of an import, so we'll fail to spot this and even if we did, we wouldn't be able to get it back out of mysql because SHOW WARNINGS only shows the last command. So let's look at the table we just created:

mysql> show create table User_\G
*************************** 1. row ***************************
       Table: User_
Create Table: CREATE TABLE  User_  (
...
) ENGINE=MyISAM DEFAULT CHARSET=latin1
1 row in set (0.00 sec)

Eh? what's going on? We asked for InnoDB, but have got a MyISAM table. Lets look at the engines available.

mysql> show engines;
+------------+----------+----------------------------------------------------------------+
  Engine       Support    Comment                                                         
+------------+----------+----------------------------------------------------------------+
  MyISAM       DEFAULT    Default engine as of MySQL 3.23 with great performance           
  MEMORY       YES        Hash based, stored in memory, useful for temporary tables        
  InnoDB       DISABLED   Supports transactions, row-level locking, and foreign keys       
  BerkeleyDB   NO         Supports transactions and page-level locking                     
  BLACKHOLE    NO         /dev/null storage engine (anything you write to it disappears)   
  EXAMPLE      NO         Example storage engine                                           
  ARCHIVE      YES        Archive storage engine                                           
  CSV          YES        CSV storage engine                                               
  ndbcluster   DISABLED   Clustered, fault-tolerant, memory-based tables                   
  FEDERATED    YES        Federated MySQL storage engine                                   
  MRG_MYISAM   YES        Collection of identical MyISAM tables                            
  ISAM         NO         Obsolete storage engine                                          
+------------+----------+----------------------------------------------------------------+
12 rows in set (0.00 sec)

Oh, so innodb has been disabled. We can fix that easily by removing skip-innodb from my.cnf.

root@cmsdb01:/var/log# grep skip-innodb /etc/mysql/my.cnf
root@cmsdb01:/var/log#

But hang on a second, that's not in the config file. What's going on? It turns out that the reason InnoDB is disabled is because of the innodb_log_file_size setting not matching the files on disk.

root@cmsdb01:/var/log# grep innodb_log_file_size /etc/mysql/my.cnf
innodb_log_file_size            = 512M
root@cmsdb01:/var/log# ls -lh /var/lib/mysql/ib_logfile*
-rw-rw---- 1 mysql mysql 5.0M 2006-12-19 18:39 /var/lib/mysql/ib_logfile0
-rw-rw---- 1 mysql mysql 5.0M 2006-12-19 18:39 /var/lib/mysql/ib_logfile1

Rumour has it that you can just stop MySQL, delete these log files and start MySQL again. I'm yet to try this as the server in question is in production use. The alternative is to change the innodb_log_file_size setting to match the file. So in summary the problems with MySQL are:

Not logging warnings anywhere useful.
Converting engine types with a warning rather than throwing an error. This can be fixed by setting sql_mode to include NO_ENGINE_SUBSTITUTION.
Starting up and disabling InnoDB when there is a problem rather than failing to start, giving a false impression that everything is working.

MySQL has not impressed me this week.

Read Comments (3)

31 March 2008

David Pashley: Daylight Saving under Debian

Unfortunately I live in the UK, where 6 months of the year, the time is GMT. Now is the time of year when I discover which of my servers don't have the right timezone configuration and show the wrong time during daylight saving. For future reference, here's how to set the timezone to Europe/London rather than UTC.

root@cms01:/tmp/openssl-0.9.8g# date
Mon Mar 31 08:23:35 GMT 2008
root@cms01:/tmp/openssl-0.9.8g# tzconfig
Your current time zone is set to GMT
Do you want to change that? [n]: y
Please enter the number of the geographic area in which you live:
   1) Africa         7) Australia
   2) America        8) Europe
   3) US time zones     9) Indian Ocean
   4) Canada time zones    10) Pacific Ocean
   5) Asia           11) Use System V style time zones
   6) Atlantic Ocean    12) None of the above
Then you will be shown a list of cities which represent the time zone
in which they are located. You should choose a city in your time zone.
Number: 8
Amsterdam Andorra Athens Belfast Belgrade Berlin Bratislava Brussels
Bucharest Budapest Chisinau Copenhagen Dublin Gibraltar Guernsey Helsinki
Isle_of_Man Istanbul Jersey Kaliningrad Kiev Lisbon Ljubljana London
Luxembourg Madrid Malta Mariehamn Minsk Monaco Moscow Nicosia Oslo Paris
Podgorica Prague Riga Rome Samara San_Marino Sarajevo Simferopol Skopje
Sofia Stockholm Tallinn Tirane Tiraspol Uzhgorod Vaduz Vatican Vienna
Vilnius Volgograd Warsaw Zagreb Zaporozhye Zurich
Please enter the name of one of these cities or zones
You just need to type enough letters to resolve ambiguities
Press Enter to view all of them again
Name: [] London
Your default time zone is set to 'Europe/London'.
Local time is now:      Mon Mar 31 09:23:48 BST 2008.
Universal Time is now:  Mon Mar 31 08:23:48 UTC 2008.

More information is available in the Debian System Administrator Manual.

Read Comments (1)

29 March 2008

David Pashley: Bad Decompression Errors in OpenSSL 0.9.8a

Recently, we rolled out a Shibboleth Single Sign On service to protect one of our services. However, we started recieving intermittant login failures, both on our automated monitoring and from customers. Curiously these failures tended to happen mostly in the evening, which isn't a peak time for us. Debugging showed that the authentication worked, but the authorisaton was failing. Shibboleth works as an apache module and daemom that protects a service, which communicates with a webservice that does the authenication processing. The log files were showing an occasional SSL error in this communcation link.

INFO shibtarget.SessionCache [43005] sessionGet: trying to get new attributes 
      for session (ID=_d0cd2f93840bb92050b28fa73d19ce4f)
INFO SAML.SAMLSOAPHTTPBinding [43005] sessionGet: sending SOAP message to 
      https://login.example.com/shibboleth/AA
ERROR SAML.SAMLSOAPHTTPBinding [43005] sessionGet: failed while contacting   
      SAML responder: error:1408F06B:SSL routines:SSL3_GET_RECORD:bad 
      decompression
ERROR shibtarget.SessionCache [43005] sessionGet: caught SAML exception 
      during SAML attribute query: SOAPHTTPBindingProvider::send() failed 
      while contacting SAML responder: error:1408F06B:SSL 
      routines:SSL3_GET_RECORD:bad decompression
ERROR shibtarget.SessionCache [43005] sessionGet: no response obtained

We didn't manage to find any suitable solutions on the internet, so we pulled out the trusty wireshark and started looking to see what was going on. We could see that the client was advertising deflate and null compression, and that the server was responding by asking for deflate compression. However the client would then claim that there was a decompression error in the servers response. This opened a few lines of enquiry. I made sure that both ends of the connection were running the same version of OpenSSL and they were both using 0.9.8a from Ubuntu Dapper. Interestingly 0.9.8a is the first version that had compression support. We found a couple of suggestions including forcing connections to be SSL2, which lacked compression or recompiling openssl without zlib support. As the former was easier, we tried that first by putting

SSLCipherSuite SSLv2:-LOW:-EXPORT:RC4+RSA

in /etc/apache2/mods-enabled/ssl.conf as suggested by Debian bug #338008, and this seemed to work for around an hour. Packet sniffing showed that it was still negotiating SSL3 including deflate compression. Clearly we had to try something else. Rather than recompile OpenSSL without zlib support, I thought I'd try upgrading the version of OpenSSL to something later in case that fixed the decompression bug. the version in Hardy is 0.9.8g, which sadly required recompiling and disabling the Ubuntu change to enable -Bsymbolic-functions during linking. Installing this on the client end didn't fix the problem, however installing it on the server end seemed to fix it. So far it's been running for 24 hours without an error, so fingers crossed that this has fixed it for good.

Read Comments (0)

27 March 2008

David Pashley: Installing java non-interactively

Installing the Sun Java packages on Debian or Ubuntu require to you accept Sun's license before you can install them. This means that it's not easy to install non-interactively, for example when using pbuilder. Fortunately the license uses debconf to check to see if you have already accepted the license. This means you can use debconf to accept the license before you install the packages. Create a file containing the following lines:

sun-java5-jdk shared/accepted-sun-dlj-v1-1 select true
sun-java5-jre shared/accepted-sun-dlj-v1-1 select true
sun-java6-jdk shared/accepted-sun-dlj-v1-1 select true
sun-java6-jre shared/accepted-sun-dlj-v1-1 select true

Then run /usr/bin/debconf-set-selections <file> and when you install the java packages, you should find it doesn't prompt for the license any more.

Read Comments (0)

12 March 2008

David Pashley: User friendly names in warnquota

By default, warnquota sends out emails with the device name in the message, which probably doesn't make much sense to most non-technical users.

Hi,
We noticed that you are in violation with the quotasystem
used on this system. We have found the following violations:
/dev/mapper/Ubuntu-home
                        Block limits               File limits
Filesystem           used    soft    hard  grace    used  soft  hard  grace
/dev/mapper/Ubuntu-home
               +- 1044404 1000000 1200000  6days    1781     0     0

You can improve this by using /etc/quotatab to assign a more meaningful name to the partition:

/dev/mapper/Ubuntu-home:user directory
/dev/mapper/Ubuntu-shared:shared area

Read Comments (1)

4 March 2008

David Pashley: SCIM ate my shift-space

I've been running Hardy on my workstation for a while and had recently noticed that I was failing to type a space after "I". I was doing it far too much for it to just be me failing to press the space bar properly, and it wasn't happening after any other letter. After a little bit of experiementing, I discovered that something was eating shift-space. What was happening was that I was failing to release the shift key quick enough after typing "I" and before I hit the space bar, so it wasn't getting passed on. Turns out that the problem was a recent update of Hardy installed SCIM, which uses shift-space as a keyboard shortcut. To turn it off, load the SCIM Setup program and go to the FrontEnd Global Setup screen and remove "Shift+Space" from the Trigger hotkey. Caused confusion for a few minutes. :)

Read Comments (0)

17 February 2008

David Pashley: Ceci n'est pas un spam

Subject: FELICITATION !!!!!  VOUS VENEZ DE GAGNER (ceci n'est pas un spam)

It's in French; of course it's a spam.

Read Comments (2)

5 February 2008

David Pashley: Outsmarting dpkg's conffile handling

dpkg has a very useful feature where if you delete a conffile (pretty much everything under /etc and a few other files) it isn't replaced when you upgrade the package[0]. This behaviour was confusing me for a while until I realised what was happening. I was attempting to reinstall a package to get the default configuration files back that had been accidentally deleted, but no matter what I tried, the files didn't exist after running dpkg. Once I figured out that dpkg had this behaviour the solution was simple; use the --force-confmiss command line argument.

root@quux:~# dpkg --force-confmiss -i /tmp/foo_2.0.0-build.14_all.deb 
(Reading database ... 33418 files and directories currently installed.)
Preparing to replace foo 2.0.0-build.14 (using .../foo_2.0.0-build.14_all.deb) ...
Unpacking replacement foo ...
Setting up foo (2.0.0-build.14) ...
Configuration file  /etc/foo/foo.xml', does not exist on system.
Installing new config file as you request.
root@quux:~#

[0] If the file didn't exist in the previously installed version, it is installed, so you get new configuration files.

Read Comments (5)

24 January 2008

David Pashley: Phisher aren't even trying

Phishers aren't even trying these days:

The following things stand out:

The date header is +0900. Suspicion rating: 2/10
The recorded log in time is in EST. The Halifax and myself are in GMT. Suspicion rating: 6/10
The recorded log in time hadn't occured by the time I get the email. Suspicion rating: 8/10
I don't bank with the Halifax. Suspicion rating: 10 million/10

Read Comments (0)

David Pashley: Child-friendly pasting in vim

If you've got various indenting and text wrapping options turned on in vim, pasting text into the editor results in screwed up results. You can get around this by turning on paste mode using :set paste and off with :set nopaste. To make things a little easier, you can use the following snippet in your .vimrc to allow you to toggle paste on and off using a single keypress:

nmap <F4> :set invpaste paste?<CR>
imap <F4> <C-O>:set invpaste<CR>
set pastetoggle=<F4>

(Warning: my vim settings have organically grown over the last 10 years, so they may not be the best or modern way of achieving an effect.)

Read Comments (3)

14 January 2008

David Pashley: ERROR 1005 (HY000): Can't create table './Database/Table.frm' (errno: 150)

If you're trying to import a dump file created using mysqldump and you get an error like:

ERROR 1005 (HY000): Can't create table './Database/Table.frm' (errno: 150)

Then you've just been bitten by mysqldump being far too stupid. The problem occurs because mysqldump includes foreign key constraints in the initial CREATE TABLE command, so if a table refers to a table that doesn't currently exist, it throws an error. mysqldump does correctly disable the contraints when inserting data into the tables. The correct way for this would be for mysqldump to create all the tables without the constraints, use ALTER TABLE to add the constraints to the tables, and then importing the data into the tables. The workaround for this problem is to use:

SET FOREIGN_KEY_CHECKS = 0;
source dump.sql
SET FOREIGN_KEY_CHECKS = 1;

Update: Someone has pointed out that it appears that mysql 5 has fixed this problem by including the above statements in the dump.

Read Comments (0)

22 December 2007

David Pashley: BREAKING NEWS

Breaking news on BBC New24. Just confirmed in the last few minutes. Very few details.... Last night, Tony Blair converted to Catholism Why is this news? Who cares? Why is the BBC treating this like it's the biggest news item of the year? Why have they rolled out Anne Widdecomme to do a phone interview? His wife is a catholic, his children are catholic, it's been on the cards for a while. He isn't in power any more. It remains to be seen if he has any relevance any more. So why does it matter what denomination he is.

Read Comments (4)

9 August 2007

Michael Janssen: UI Peeves: Locking, Proximity and Purpose

I have had, on occasion in the past, a reason to use the wonderful TortoiseSVN software. It is, by all standards, a really great solution for someone wanting to use SVN in a MS Windows environment, and uses the official svn client in order to leverage open source as much as possible. Recently I have been using it a bit more as I work at Honeywell as a programmer. I've come across a couple things that bug me about the interface.

The first is that some actions inexplicably lock the place that initiates that action. The best example I have of this is when I am looking at the window for an Update, and want to see what happened in the log, I click the button to open the log, and I can't use the window while the log downloads. Sometimes this can be an issue because the server is pretty slow at producing logs for some reason (it's beyond me, and NotMyProblem(tm) because I'm not in charge of the admin). This makes it irritating because I can't inspect other files while the log is downloading. Strangely, the server actually will return other requests quickly, like the request needed to see the diff between the working copy and the tree. It would be much better if the windows were separate - they're not related to each other in effect, so the locking of the initial window is completely useless. I could have called up the log from another action, that would leave the update window perfectly usable, so I know it's possible.

The second problem that I have run across more than once relates to the proximity of actions in the right click menu while committing a change. Take a look at the screenshot of this menu.

TortoiseSVN commit context menu

There have been many times when using this window that I realize that I forgot to add a file that I need to commit, so I right click on the file and move down to the "Add" menu item, only to miss and click on the "Delete" item. "Delete", when used on a unversioned file, performs a windows delete, causing me to curse at the (now missing) entry in the commit log and go running to the recycle bin to retrieve this temporarily lost file. These two options in the context menu are entirely too close to each other. The "Delete" and "Add" actions have completely opposite meanings and effects, and having the program do exactly the opposite of what you expect is what I consider to be a Very Bad Thing. I would move the "Add" action up to the top of the list (it is, by far, the most common reason for me to use the context menu), or at least place another menu item in between.

I'm feeling like I'm being very unhelpful here, complaining about these issues in an open source project and not producing a patch to fix these issues. I may produce one in the near future - unfortunately when I'm not at Honeywell, I don't really have a Windows development environment setup.

Comments: (1) Trackbacks: (0)

Comments

JD: The reason diffs are quick is because svn keeps a copy of the original file in the checkout, so diffs and reverts don't need to hit the server.
#1 2007-08-09 00:41 (Reply)

Trackbacks

No Trackbacks

Next.

Previous.